Asymptotic Quadratic Convergence of the Two-Sided Serial and Parallel Block-Jacobi SVD Algorithm

This report is devoted to the proof of the global convergence and asymptotic quadratic convergence of the serial and parallel two-sided block-Jacobi SVD algorithm. In the serial case, one pair of the off-diagonal blocks with the largest weight given as the sum of squares of Frobenius norms is annihilated. In the parallel case, using the greedy implementation of dynamic ordering and having p processors, p pairs of the off-diagonal blocks with largest weights and disjunct block row and column indices are annihilated in each parallel iteration step. 1 Serial two-sided SVD algorithm It is assumed in this report that the singular value decomposition is computed for a square matrix. Hence, when the original matrix is of size m × n, m ≥ n, compute first its QR decomposition and then apply the SVD algorithm to the n× n factor R. Let us divide a square matrix A of order n into a w×w block structure with w blocks in each block row (column). Denote by AIJ the (I, J)th block of size `× `, ` = n/w. Hence, there are w(w − 1) off-diagonal blocks in A. Let us assume that, at the initialization step, all diagonal blocks of A were diagonalized by a series of unitary, two-sided transformations. Diagonal blocks remain then diagonal during the whole computation. In the kth step of the two-sided serial block-Jacobi SVD method, let us define weights for ∗Mathematical Institute, Slovak Academy of Sciences, Bratislava, Slovak Republic, email: [email protected] †Department of Communication Engineering and Informatics, University of Electro-Communications, Tokyo, Japan, email: [email protected] ‡Department of Computer Sciences, University of Salzburg, Austria, and Mathematical Institute, Slovak Academy of Sciences, Bratislava, Slovak Republic, email: [email protected] off-diagonal blocks with symmetric block indices (I, J) and (J, I), I 6= J , by w (k) IJ ≡ ‖A (k) IJ ‖F + ‖A (k) JI ‖F . (1) To optimally reduce the off-diagonal Frobenius norm, the pair of off-diagonal blocks with the maximal weight will be eliminated. Let these off-diagonal blocks have block indices (Xk, Yk) and (Yk, Xk), i.e. w (k) XkYk = max I 6=J w (k) IJ . Notice that, contrary to the EVD of Hermitian matrices, choosing two off-diagonal blocks with maximal weight for annihilation is not equivalent to choosing the off-diagonal block A (k) SkTk with the largest Frobenius norm together with the block A (k) TkSk . In fact, one can easily have w (k) SkTk < w (k) XkYk , so that the off-diagonal block with the largest Frobenius norm is not eliminated. The annihilation is performed by a two-sided unitary transformation (U )AV (k) = A, where the n× n unitary matrices U (k) and V (k) are the matrices of local left and right singular vectors, respectively, embedded into the identity matrix In of size n. Four blocks of U (k) and V , each of size `, that are different from blocks of In can be chosen so that ( U (k) XkXk U (k) XkYk U (k) YkXk U (k) YkYk )H ( A (k) XkXk A (k) XkYk A (k) YkXk A (k) YkYk )( V (k) XkXk V (k) XkYk V (k) YkXk V (k) YkYk ) = ( A (k+1) XkXk 0 0 A (k+1) YkYk ) , (2) whereby the diagonal blocks A (k+1) XkXk and A (k+1) YkYk are square, diagonal matrices of order ` with non-negative diagonal elements (local singular values). Let us define Ũ (k) ≡ ( U (k) XkXk U (k) XkYk U (k) YkXk U (k) YkYk ) , Ṽ (k) ≡ ( V (k) XkXk V (k) XkYk V (k) YkXk V (k) YkYk ) , (3) and à ≡ ( A (k) XkXk A (k) XkYk A (k) YkXk A (k) YkYk ) , Σ ≡ ( A (k+1) XkXk 0 0 A (k+1) YkYk ) . (4) Since Eq. (2) is essentially the SVD of the matrix Ã, the matrix Ũ (k) and Ṽ (k) is the unitary matrix of left and right singular vectors of Ã, respectively. To prove the global convergence of the parallel two-sided block-Jacobi SVD method, let us define the square of the off-diagonal Frobenius norm of A by ‖off(A)‖F ≡ ∑ I 6=J ‖A IJ ‖F . (5)